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(57) Abstract 

An interactive audio-visual (A/V) 
transceiver is advantageously coupled 
to a television and/or telephone (T/T) 
cable, a TV. a video recorder (VCR), 
and other A/V devices. The A/V 
transceiver switches data berween a r 
program/ service provider and the con- | 
cected A/V devices. In one embodi- j 
rnent- the transceiver includes three pri- j 
mary module s, a main module includ- 
ing a CPU, a system bus, system mem- r 
. cry, an infra-red (IR) control unit, an j 
audio- visual bus, an A/V decoder, an j 
A/V processor, and an .A/V encoder, an t 
.A/V connect module including a aum- 
, ber of mner/demodulatcrs and a switch, 
and an optional CD ROM module. The 
A/V transceiver hardware is comple- 
mented with an operating system and 
software program which supports the 
functions provided in the .A/V user in- 
i;n"ace. Additionally, a remote con- 
trol .i*vice is provided to cemmum- 
care with the A/V transceiver to interac- 
tively cranage selection or program and 
service sources, selection Droeram and 

se-v-cc ocnacs from any selected source, viewing of selected program offerings, and interaction with selected service ocenngs. ine 
-moie control" device is advantageously Dtovided with a basic A/V controi bunon group, an interactive control burton group, an auxdiary 
control buToo zrouo and a numeric ke'v oad a facilitate coctrol of see transceiver, iae interactive control burton group induces an imo 
button a Us: bunco! a categories bctton. a pU burton, a mark button, a jump bunoa. and a pointing device consisting ot up, oowd. lea. and 
-ight arrow burtons, and a center setecc button. 
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AN AUDIO-VISUAL SYSTEM FOR SELECTIVELY 
VIEWING AND INTERACTING WITH PROGRAMS AND 
SERVICES FROM A NUMBER OF PROGRAM/SERVICE SOURCES 

RELATED APPLICATIONS 

The present application is related to the following 
contemporaneously filed applications: 

(a) Serial No. , entitled "Method And Apparatus 

For Controlling An Audio-Visual System For Selectively Viewing And 
Interacting With Programs And Services From A Number of 
Program /Service Sources"; 

(b) Serial No. , entitled "Methods And Apparatus 

For Managing Selection Of Audio-Visual Program And Service 
Offerings"; 

(c) Serial No. , entitled "Methods And Apparatus 

For Managing Viewing Of And Interaction With Audio- Visual Program 
And Service Offerings". 

BACKGROUND OF THE INVENTION 
1. Field of the Invention; 

The present invention relates to the field of audio-visual 
systems. More specifically, the present invention relates to a system for 
selectively viewing and interacting with programs and services from a 
number of program /service sources, a control device for controlling the 
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The combination of computer technology with television 
(TV) and audio-visual (A/V) systems, has fostered the development of 
multi-media interactive entertainment systems. The combination of 
computer hardware and software with audio-visual systems has brought 
new forms of art and entertainment into being. CD read only memory 
(ROM) systems coupled to personal computers permit interactive video 
computer simulations, unique self-paced learning environments and 
interactive "movies", not possible in earlier television systems (See, 
"The Revolution Starts Here", Newsweek, page 42 (January 18, 1993)). 
is know* that consumers desire interactive television and multi-media 
applications for home use. One of the biggest challenges for an 
interactive television service is the design of an interface that is easy and 
fun to use by average consumers. 

In the computer industry, a variety of graphic user 
interfaces have been developed to facilitate human interaction with 
computer systems. Many display systems utilize metaphors in the design 
of the interface as a way of maximizing human familiarity, and 
conveying information between the user and the computer. It is well 
known that designing around a familiar metaphor helps reduce human 
learning time (See for example, Patrick Chan, "Learning Considerations 
In User Interface Design: The Room Model", Report CS-84-16, University 
of Waterloo Computer Science Department, Ontario, Canada, July, 1984, 
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SUMMARY Q -P THF TNVENTION 



/ The present invention provides methods and apparatus for 

presenting an improved audio-visual user interface for selecting and 
displaying cable television or other audio-visual programs, as well as 
controlling various audio-visual devices and interactive services. The 

C present invention assumes a service provider provides cable television 
and /or telephone (T/T) service to users via a T/T cable, including a 
digital channel of program/ service listings, at least one digital back 
channel (from the user's home to a central file server), a number of 
analog TV channels, a number of digital pay-per-view channels, and 
/•-. other interactive services, transmitted from remote storage devices such 
as digital file servers. Under the present invention an interactive audio- 
visual (A/V) transceiver is advantageously coupled to a user's 
television, video tape recorder, and the T/T cable. In one embodiment, 
the transceiver includes three primary modules. A main module 

A C~ includes a central processing unit (CPU) coupled over a digital system 
bus to system memory and, preferably, an infra-red (HO or similar 
wireless control unit. The main module is further coupled over an 
audio-visual bus to an A/V decoder, an A/V processor, an A/V encoder, 
a TV, an A/V connect module connected to the T/T cable, and to one or 
more VCRs and other A/V devices. The main module may also be 
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providing the same functions (such as a mouse, thumb stick or touch 
pad). While viewing the TV, a user may obtain additional information 
on a current program by depressing the info button, and obtaining more 
detailed information using the pointing device. By depressing the list 
button on the remote control device, the transceiver displays a program 
listing of the current programs available for viewing. Through the use 
of the pointing device, viewers can scroll up and down the program 
listing or view a highlighted program in full screen by pressing the select 
button. By pushing the right or left arrow buttons on the pointing 
device, program listings may also be viewed for different hours, days and 
up to several weeks in advance through the present invention's use of 
data interleaving. The depression of the categories button on the remote 
control device results in the display of a categories menu bar on the TV 
screen, which includes categories such as "air*, "sports", "news" and 
"favorites". The selection of one of these categories results in the display 
of only programs within that category. Moreover, through the 
depressions of both the list button and the categories button, a program 
listing is displayed of the currently available programs within the 
selected category. Through the use of the arrow buttons on the pointing 
device, category listings for different times and dates up to several weeks 
in advance may also be viewed. The selection of the favorites category 
results in the display of currently available favorite programs, including 
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' rewinding, or fast-forwarding audio-visual programs on connected A/V 
playback devices, a numeric keypad for dialing channel numbers, 
confirming financial transactions with personal identification numbers, 
an optional microphone with talk button for supplementing the present 
f user interface with direct voice commands through the use of voice 
recognition technology, a pay-per view interface, and a home shopping 
interface, as well as other features described herein. 
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FIGURE 4b illustrates the first embodiment of the remote 
control device of the present invention shown in Figure 4a with the 
sliding panel in an open position. 

FIGURES 5a and 5b illustrate a second embodiment of the 
remote control device of the present invention with no sliding panel 
and additional features. 

FIGURE 6 illustrates the present invention's full ccre^n 
viewing function as viewed by a user with a graphic overlay panel which 
includes the current channel number as well as the channel logo and 
identifier. 

FIGURE 7 illustrates the present invention's info function 
showing the first page of the basic information track ( M info-track n ). 

FIGURE 8 illustrates the present -invention's info function 
with a second page of basic information, obtained by using the right 
arrow button on the remote control device. 

FIGURE 9 is an additional illustration of the info function 
of the present invention in which the user has requested a third page of 
information using the remote control device. 
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starts at 8:10 p.m., using the right arrow button of the remote control 

device. 

FIGURE 15 is an additional feature of the list function of the 
present invention where the user marks the highlighted program with 
the mark button of the remote control device. 

FIGURE 16 illustrates the selection of an alternate hourly 
time slot beginning at 9 p.m. using the right arrow button of the remote 
control device. 

FIGURE 17 illustrates the selection of an alternate day for 
the program/service listing by depressing the right arrow button of the 
remote control device. 

FIGURE 18 illustrates displaying of a record panel 
confirming the title and length of the program to be recorded, along with 
a confirmation of which VCR and tape to record on, by depressing a 
select button in the remote control device. 

FIGURE 19 illustrates the present invention's use of the 
select button on the remote control device while the program /service 
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FIGURE 24 further illustrates the categories function 
wherein the "sports" category is highlighted by depressing the right 
arrow button on the remote control device. 

FIGURE 25 illustrates a representative television image in 
the case where the "sports" category has been selected by depressing the 
select button on the remote control device. 

FIGURE 26 is a further illustration of a representative 
television image in which the "sports" category has been selected., and an 
alternate sports channel has been further selected by a user through the 
use of the up arrow button on the remote control device. 

FIGURE 27 illustrates a sports program listing in which the 
list button has been depressed after the "sports" category has been 
selected. 

FIGURE 28 illustrates the categories function in conjunction 
with the list function. 

FIGURE 29 further illustrates the categories and list 
functions when the down arrow button is depressed on the remote 
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FIGURE 34 illustrates the pix function of the present 
invention in which a new PIP window has been highlighted and is 
displayed in a larger format in the center of the display screen. 

FIGURE 35 is a further illustration of the present 

device a PIP window showing a pay-per-view program has been 
highlighted and is displayed in the center portion of the screen. 

FIGURE 36 illustrates the menu function which permits 
users to press a menu button to switch the television display to other 
audio-visual devices coupled to the transceiver, such as VCR, CD, on- 
Une services, telephone, etc. For example, once VCR is selected, with the 
ri-ht arrow button, the TV displays an image from the videotape 
clently in the VCR and the user can press the play button on the 
remote to playback a previously recorded program on the video tape. 

FIGURE 37 illustrates one embodiment of the present 
invention for selecting a pay-per-view channel offering entitled "Jeffs 
World" through the use of the select button of the remote control dev.ce. 



FIGURE 38 illustrates a preview feature of pay-per- 
under the above illustrated embodiment. 



view 
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FIGURE 45 illustrates a menu permitting the user to select 
various shops available on the home shopping service using the select 
button on the remote control device under the above illustrated 
embodiment. 

FIGURE 46 is a further illustration of the home shopping 
service in which a cookware product is highlighted under the above 
illustrated embodiment. 

FIGURE 47 is a further illustration of the home shopping 
service in which a mini-espresso machine is highlighted, and can be 
selected for ordering with the select button on the remote control device 
under the above illustrated embodiment. 



FIGURE 48 is an information screen provided to the user 
after selecting a mini-espresso machine for ordering under the above 
illustrated embodiment. 

FIGURE 49 is an order screen in which the user, using the 
numeric keypad on the remote control device, enters a personal 
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x,n T AT!ON i-iMP ^MCNn.ATOBE 

The detailed descriptions which follow are presented largely 
in terms of interface display images, algorithms, and symbolic 
.presentations of operations of data bits within a computer memory. 
These algorithmic descnptions and representations are the means used 
by thos e s«Ued in the data processing arts to most effectively convey the 
substance of their work to others skilled in the art. 

A„ algorithm is here, and generally, conceived to be a self 
consistent seance of steps leading to a desired result. These steps are 
those recurring physical manipulations of physical quantities. Usually, 
th ough not necessarily, these quanuties take the form of electrical or 
magnetic signals capable of being stored, transferred, combined, 
compared, displayed and otherwrse manipulated. It proves conventent 
at times, prindpally for reasons of common usage, to refer to these 
signals as bits, values, elements, symbols, characters, images, terms, 
numbers, or the like. It should be borne m mind, however, that of 
.hese and similar terms are to be associated with the appropriate physrca, 
quantities and are merely convemen, labels applied to these quants. 
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^ufactured by .he Assise, App!e Compute, Inc., as we,l as Oner 
m anu f actu rer s of colter and computer — d — 



systems. 
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r.^-nPTal Svsf™ rnn figuration 



/ The multi-media interactive television system of the 

present invention is illustrated in schematic form in Figure 1. As shown 
in Figure 1, a cable or telephone service provider 50 provides cable 
television or telephone (T/T) services over a T/T cable 52 to a plurality of 

tr users coupled to the cable or telephone system, as is known. Although in 

" this Specification reference is made to a cable television or telephone 
system, it will be appreciated by one skilled in the art that the present 
invention may be used in conjunction with a variety of other electronic 
transmission systems including satellite service systems, microwave 

/"; systems, fiber optic, and radio frequency (RF) systems. 

As illustrated in Figure 1, the T/T cable 52 is coupled to an 
audio-visual transceiver 54 which comprises a number of separately 
identifiable modules. The transceiver 54 is intended to be located in 
proximity to and coupled to a VCR 56, a television (TV) 58, as well as one 
/V or more optional audio-visual devices 57 such as additional VCRs, laser 
disc players, camcorders, stereos, various storage devices, telephones, 
faxes and answering machines, as shown. Moreover, it is contemplated 
that transceiver 54 may be directly incorporated into the VCR 56 , the TV 
58, or one the other A/V devices 57. As will be described below, a remote 
) - control device 60 communicates with the transceiver 54 preferably 
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or the other A/V devices 57, and couples those signals and data to the 
main module 62 through the video decoder 74. The A/V connect 
module further switches audio-visual signals and data received from the 
main module 62 through the video encoder 78, and sends them back out 
to the T/T cable 52, the VCR 56, the other A/V devices 57 and/or the TV 
58. The A/V connect module 66 includes a switcher 67 and one or more 
programmable tuners /demodulators 69, wherein one 
tuner/ demodulator reads and displays a current program from one of the 
channels received over the T/T cable 52, and additional 
tuners /demodulators (or the same tuner/ demodulator, used in 
alternation) are used to read and display data from the side-band 
channels in picture-in-picture (pip) windows. In addition, the A/V 
connect module 66 may include descrambling circuitry (not shown) to 
descramble premium programs received over the T/T cable 52. It is 
further contemplated that the A/V connect module 66 provides a graphic 
overlay function that superimposes an A/V signal from the video 
encoder 78 against another A/V signal from the T/T cable 52, the VCR 56, 
or the other A/V devices 57 allowing both signals to be simultaneously 
displayed on the TV 58, the VCR 56 or the other A/V devices 57. Finally, 
the A/V connect module 66 can be used to transmit data such as order 
information to the cable (T/T) service provider 50 over a back channel 
102 described below. 
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re-arrange digital data into coherent audio-visual displays. An optional 
A/V memory module 75 also coupled to the A/V bus 73 can be used to 
store A/V data before or after processing by the A/V processor 77 (using 
components such as video random access memory (VRAM) to hold, for 
example, a frame buffer equivalent to one 640 x 480 x 8-bit color image in 
memory). 

The A/V encoder 78, coupled to the A/V bus 73 through the 
memory and bus controller 80, is used to convert digital A/V data from 
the A/V processor 77 or the optional A/V memory 75 into analog audio- 
visual signals which are routed to the TV 58, the VCR 56 or the other 
A/V devices through the A/V connect module 66. It is also 
contemplated that the A/V encoder 78 may be used to encode other data 
such as order information sent by the CPU 63 through the memory and 
bus controller 80 and the system bus 64 and transmit them to the service 
provider 50 through the A/V connect module 66 and a back channel 102 
described below. 

The CPU 63 is also coupled over the system bus 64 to a 
system memory 65 including both volatile and non-volatile memory 
components. The non-volatile part of system memory 65 includes read- 
only memory (such as ROM), which is used to store an operating system 
and playback software, fonts, sounds and the like used in the present 
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interface (SCSI) for accessing digital storage devices such as hard disks, a 
modem for exchanging digital data over telephone lines, or a serial port 
for controlling other devices over a wired connection. Another module 
presently being contemplated is a remote beeper button for locating the 
remote control device 60. Many consumers often misplace their remote 
control devices, and it is contemplated that the present invention may 
provide a function wherein a speaker on remote control 60 would beep 
while the beeper button is pressed until the user locates the remote 
control. 

Referring now to Figure 3a, there is shown one possible 
electronic spectrum of signals provided by the cable T/T service provider 
50 over the T/T cable 52 to the transceiver 54. As illustrated in Figure 3a, 
the spectrum includes at least one digital program listing channel 100, a 
plurality of back channels 102, and a plurality of standard analog TV 
channels 103. In addition, and as will be described more fully below, the 
spectrum includes pay-per-view digital channels 106, and further 
additional definable digital channels 108 offering a variety of interactive 
services, in addition to the general purpose or special channels or 
services particular to the specific application of the present invention. 



As illustrated in Figure 3b, the digital program listing 
channel 100 provides data representing daily and weekly program 
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I whereafter the sections that are most relevant to the users are stored in 
the system memory 65. By interleaving the current day's data with the 
weekly data, the daily data, the CPU module 62 can update the system 
memory 65 periodically and still provides quick access to the viewer, 

s without having to store all the received program /service information, 
thereby reducing the amount of system memory 65 required and the 

^associated hardware cost. 

Referring cnce again to Figure 3a, the back channels 102 are 
used to engage in a variety of transactions, such as ordering products, 
/ 0 home banking services and pay-per-view movies, as will be described 
more fully below. In operation, any request by a user to view a pay-per- 
view movie or order a product is transmitted to the service provider 50 
over at least one back channel 102 through the T/T cable 52. It is 
contemplated that as channel capacity increases, so will the number of 
^Tback channels, allowing for broader band two-way communications such 
as telephone conversations or interaction with on-line networks. 



Referring now to Figures 4a, 4b, and Figures 5a and 5b, two 
alternative embodiments of the remote control device 60 are shown in 
further detail. It is expected that the model shown in Figures 5a and 5b 
would be more commonly used. As illustrated in the figures, both 
embodiments of the remote control device 60 are comprised of four 
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a^d as will be described below, the user may desire to watch only sports 
programs, only news programs, or the like. As will be disclosed, the 
categories button 140, and the list button 138 operate in concert to 
provide listings of selected categories, such as sports or news programs, 
on the TV 58. The mark button 142 permits the user to mark programs 
for reminders, later recall, or switching between programs which have 
been selected using the mark button 142. A jump button 132 lets the user 
switch to the program that was last marked, then to the program that was 
marked before that, and so on, until all marked programs have been 
shown, looping back to the last marked program again. As shown in 
Figures 5a and 5b, the jump burton 132 and the mark button 142 are 
placed side by side together on the dividing line between the first and 
second control button groups 115 and 134 to emphasize their 
interrelation. 

Also provided with the interactive control button group 134 
is a pointing device consisting of an up arrow button 145, a down arrow 
button 146, a left arrow button 148, and a right arrow button 150. For 
example, if the user is viewing a program /service listing through the use 
of list button 138, and desires to scroll up or down within the listing, the 
up arrow button 145 and the down arrow button 146 are utilized. In 
addition, a center select button 155 is provided for permitting the user to 
select one of several choices, represented by words or icons in graphic 
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being programmed to record the particular show highlighted in the 
program listing. Pressing the menu button 172 displays icons for all the 
A/V devices 57 currently connected to the transceiver 54 and allows 
users to select another device with the arrow buttons. Pressing the center 
select button 155 displays the output of that device and lets the user 
control that device. 

The fourth control button group is the numeric keypad 
panel 176, which permits the user to directly enter channel numbers to 
be viewed on the TV 58 by pressing a single, double or triple digit 
number, followed by the ok button 178 (Fig. 5a). For the remote control 
device 60 shown in in Fig 4b, the user would depress the select button 155 
after entering values using the keypad 176. In addition, the numeric 
keypad 176 may be used by the user to enter a personal identification 
number ("PIN"), to be transmitted to the service provider 50 through the 
A/V connect module 66 for payment of pay-per-view movies, products 
and the like. 

In addition to the control button groups, both embodiments 
comprise a transmitter (not shown) for transmitting signals to the audio- 
visual system. Additionally, the model illustrated in Figures 5a - 5b is 
further provided with a talk button 181, a microphone 179, a speaker 182 
and a receiver (not shown). The talk button 181 when depressed allows 
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Pyll Screen Viewing Function 

Referring now to Figure 6, there is shown a representative 
full screen audio-visual program displayed on the screen 180 of the TV 
58. By depressing the channel up/down buttons 130 on the remote in 
Figures 4a and 4b, (or by depressing the up and down arrow buttons 145 
and 146 on the remote control device in Figures 5a - 5b), users can switch 
channels as is commonly done with regular TV remotes. As the user 
selects a channel to view using the remote control device 60, a graphic 
overlay panel 185 is displayed in the lower right hand portion of screen 
180. As illustrated, the graphic overlay panel 185 includes the channel 
number currently being viewed 186 along with the channel logo and 
identifier 188. Additionally, a marking identifier 189, such as a check 
mark, is also displayed, as will be described, to indicate that the program 
has been marked through the use of the mark button 142 previously 
discussed with reference to the remote control device of Figures 4a, 4b, 
and Figures 5a and 5b. 
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, button 150, and the down arrow button 146, of the remote con.ro! devtce 
60 can be depressed to cause addi.ie.na! information to be depressed. An 
up arrow 205 is also displayed, bu. is not highlighted, .0 indicate that no 
additional information will be displayed if the up arrow button 145 is 

^depressed. 

Referring now to Figure 8, which illustrates a new graphic 
overlay panel 190 displayed in response to the user having depressed the 
right arrow button 150 on the remote control device 60 while using the 
information function. The transceiver 54 pr^s_ad£toal 
h infor mation on the currently viewedj rogTam. As illustrated, an 
additional page of information on the program currently viewed is 
displayed. For example, additional guests on the show "Showbits This 
Month" include "Madeline, Harold Black and Harmer". Referring now 
to Figxure 9, which illustrates another new graphic overlay panel 190 
K displayed in response to the user having depressed the right arrow 

button 150 again. A third page of information is displayed on the screen 
180 relating to the currently viewed program "Showbits This Month". It 
will be noted that the currently viewed television program ("Showbits 
This Month" in this example) continues to be viewed in the main 
2* portion of the screen 180, while different graphic overlay panels 190 are 
superimposed over the lower third portion of the screen 180. 
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information relating to John Smith is provided in the graphic overlay 
panel 190. It will also be noted that in Figure 10, the down arrow 210 is 
highlighted, indicating that yet another information track is available. 

Still referring to Figure 10, it will be noted that a down 
arrow 210 is shown. By depressing the down arrow button 146 on the 
remote control device 60 once again, an Info Mart track information 
track as shown in Figure 11 is displayed by CPU 63 in the graphic overlay 
panel 190. As illustrated, the Info Mart information track provides the 
viewer with the opportunity to purchase products, services, programs 
...and *e like related to the program which is currently being viewed (in 

the present example, the baseball game "Saint Louis at San Francisco"). 

The information track disappears when the info button 136 is pressed 

again. 

While the information has been described with the left and 
arrow arrow buttons 148 and 150 causing different pages of an 
information track to be displayed, and the up and down arrow buttons 
145 and 146 causing different information tracks to be displayed, it is 
contemplated that the information displayed may be related to each 
other in other application dependent manners. It is further 
contemplated that this information function can also be used with 
audio-visual programs or services other than TV shows, in particular 
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T.ist Function 



Referring now to Figure 12, the list function of the present 
invention will be described. While viewing a television program 
displayed on the TV 58, the user may depress the list button 138 on the 
remote control device 60 to obtain a program /service listing for the 
current date and time during which the user is watching television. As 
shown in Figure 12, upon depressing the list burton 138, a 
program/service listing 220 is displayed for the current date and time at 
which the viewer has depressed the list button 138. As illustrated, 
programs/services currently available for viewing and interaction are 
identified by their channel numbers, channel name identifiers (for 
example, Channel 10, NTV), and by titles of the programs /services. In 
the event that the program/service listing includes two programs for a 
given time interval, then the names of both programs are displayed. For 
example, in Figure 12, Channel 10 is NTV which beginning at 8:00 p.m. is 
showing a program entitled "Rock Today", and beginning at 8:10 p.m. is 
showing a program called "Party Time". In addition, as shown in Figure 
12, a picture-in-picture window 250 continues to display the currently 
viewed program which the user was last viewing. Accordingly, the user 
may continue to view the currently selected program, and the current 
program/service listing simultaneously. By pressing the select button 
155 in the current time slot, the user can view the currently highlighted 
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higher channel numbers previously displayed at the bottom part of the 
screen 180 will consecutively disappear. As illustrated in Figure 14, 
Channels 7 and 8 are now displayed, and Channels 19 and 20 are no 
longer displayed to the user. If the up arrow button 145 is now pressed 
instead, the highlighting will move down towards the higher channel 
numbers as illustrated in Figure 14. Notably, movement of the 
program/service listing and the highlighting in the present invention 
are specifically calculated to conform with an average person's 
expectations in using a television remote control. While the list 
function has been described with the down arrow button 146 causing 
highlighting to move towards the lower channel numbers, and the up 
arrow button 145 causing highlighting to move towards the higher 
channel numbers, it will be appreciated that the correspondence between 
the arrow buttons 145 and 146 and the highlighting movement may be 
implemented in other ways as best suits the anticipated user of the 
interface system. 

Skipping now to Figure 16, a further feature of the list 
function of the present invention is illustrated. In the event the user 
depresses the right arrow button 150, the CPU 63 displays the next hourly 
time slot listing on the screen 180. In the example of Figure 16, the time 
slot for 9:00 - 10:00 p.m. is illustrated. It will be appreciated that, had the 
user depressed the left arrow button 148 three times, then the 
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Referring now to Figure 18, which illustrates a 
program/service listing displayed in response to the user having 
continuously depressed the right arrow button ISO on the remote contro. 
device 60. The exemplary program/service listing displayed rs for 
Saturday, October 17 beginning a, prime time (6:00 p.m.). The up and 
down arrow buttons US and 146 on the remote contro. 60 m then used 
to highlight the program "Specials Tonight". Assume now the user 
desires to record the exemplary program "Specials Tonight". By pressmg 
the record button 162 on the remote control devce 60, a record panel 2=9 
is displayed by the CPU 63 on the screen 180, confirming the title and 
length of the program to be recorded, along with a highlighted select 
("bull's eye") icon as shown in Figure 18. It is contemplated that other 
functions can b. included in the record panel 259, such as a function for 
selecting different VCRs or other A/V recording devices, a function for 
select different tapes or reading media, and/or a function for selectmg 
how often the recording is to take place (once, weekly, daily). Once the 
select button 155 is pressed, the program listing is redisplayed without 
the record oanel 259. A record icon 260 is displayed on the screen 180, as 
shown m Figure 19. The "Specials Tonight" program which will begm 
on Saturday, October 17, will then be recorded automatically. In the 
even, the viewer changes his mind and does no, desire to record the 
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M?T k and TvT"? Functions 



Skipping now to Figure 15, the mark function of the present 
invention and its operation will be described. Shown in Figure 15 is a 
program listing reflecting Channel 10 (NTV) having been selected using 
the up and down arrow buttons 145 and 146 on the remote control device 
60. Assume now the current time is 8:08:30 p.m. In accordance with the 
teachings of the present invention as described earlier, the program 
"Rock Today" would be highlighted on the screen ISO. The program 
"Party Time" may be highlighted by the viewer by pressing the right 
arrow button 150. The mark function may then be initiated by pressing 
the mark button 142 on the remote control device 160. As illustrated in 
Figure 15, a mark identifier 252 is displayed adjacent to the selected 
program (in the present example, "Party Time"). Skipping again to 
Figure 20, a reminder box 254 is displayed on the screen 180 at the time 
the program "Party Time" begins (in the present example, 8:10 p.m.), 
regardless of what channel is currently being viewed then by the user on 
TV 58. It should also be noted that programs playing live in a current 
time slot can be marked while displayed in full screen by depressing the 
mark button 142. This causes a mark identifier 189 to be displayed in the 
graphic overlay panel 185, as shown in Figures 6 and 21. 
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rptfgnries Function 



Referring now to Figures 22 - 32, the categories function of 
' ' the present invention will be described. To initiate the categories 

function of the present invention, the categories button 140 is depressed 
on the remote control device 60. As illustrated in Figure 22, the CPU 63 
^ displays a categories graphic overlay panel 300 which is superimposed 
over the currently viewed program on screen 180. A plurality of 
categories are identified by representative icons. The first category which 
is selected by default is the "all" category 302, and is highlighted around 
the border. The "all" category 302 refers to all currently available 
• > programs/services being transmitted by the service provider 50. By 
depressing the right arrow button 150 or left arrow button 148 on the 
remote control device 60, other categories may be highlighted. For 
example, by depressing the right arrow button 150, the next category 
"favorites", as indicated by a star icon 305, is highlighted (See Figure 23). 
ft Similarly, by depressing the right arrow button 150 again, a movies icon 
310 is highlighted. By depressing the right arrow button 150 another 
time, a sports icon 315 is highlighted (see Figure 24), and so on. 
Additional categories may include a news icon 312 and a music video 
icon 313. However, it will be appreciated that many other categories can 
£ & be displayed: By repeatedly pressing the right arrow button 150, the user 
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and the channel identifier 188 (in the example of Figure 26 "Sportstime") 
in the graphic overlay panel 185. 

Referring now to Figure 27, if the user has selected the 
category of sports (icon 315), and depresses the list button 138, a 
program/service listing 360 of currently available sports programs is 
displayed on screen 180. It will also be noted that the currently viewed 
sports program will continue to be displayed in a picture-in-picture 
window 365. As previously discussed, with respect to the list fusion, 
programs which are shown during half hour intervals (See Figure 27 
"Windsurfing" and "Basketball") are identified and may be individually 
selected using the remote control device 60. 

Referring now to Figure 28, which illustrates a program 
listing with the sports icon 330 in the category overlay panel highlighted, 
displayed in response to the user having pressed the list button 138, and 
then presses the categories button 140. Assume now the user desires to 
break dovm the main sports category into finer sub-categories such as 
baseball, football, basketball, hockey, etc. By depressing the down arrow 
button 146 on the remote control device 60, the user causes the CPU 63 to 
select a default sport sub-category (baseball in this example) and display a 
plurality of sports sub-category icons against the categories overlay panel 
300 as illustrated in Figure 29. The default sports sub-category icon 315 is 
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program listing 362 of programs most frequently watched by the user as 
determined by the CPU 63 from previous monitoring of the user's 
viewing habits. In the normal course of operation, the CPU 63 monitors 
programs which are most frequently viewed and compiles a prioritized 
list of those frequent programs for each time slot, which list is stored in 
the rewritable non-volatile part of the system random access memory 
(such as SRAM) 65. 

In the presently preferred embodiment, if a viewer views a 
television program for more than 10 minutes, the current program and 
channel identifiers are compared to frequent programs already stored in 
system memory 65 for that particular time slot, and an updated list of 
frequent programs is created by CPU 63 and stored in the system memory 
65. In the event there are no frequent programs stored in the system 
memory 65 for any particular time slot, then a general list of most 
frequently watched programs is retrieved from the system memory 65. It 
is anticipated that the majority of frequent programs will be stored in 
time slots corresponding to prime time, or during daytime television 
hours. 

By pressing the right arrow button 150, the user can move 
the highlighting from the "frequent" programs sub-category icon 318 to 
the "marked programs" sub-category icon 319, as shown on Figure 32. 
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While the present invention has been described with the 
categories function having different effects on the displayed image when 
used during full screen display and when used in conjunction with list 
function, it will be appreciated that the present invention also may be 
practiced with the categories function having consistent effect on the 
displayed image in either situation, as desired. 
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perceives each of the 12 small picture-in-picture windows as sequential 
representative images of every N frames per window. However, the 
picture-in-picture window 375 represents a full motion display of the 
currently selected program, provided that a second TV tuner 67 is 
included within the A/V connect module 66. 

Additionally, in the preferred embodiment of the present 
invention, it is contemplated that the user is able to view the pix display 
381 by category by pressing the categories button 140 and selecting a 
particular category, such as favorites or movies. For example, the pix 
display 381 illustrated in Figure 33 is comprised of only favorite 
programs, as the user had previously selected the favorites category. Had 
the user selected the "marked" sub-category, only marked programs 
would be displayed on the pix display 381, allowing users to easily add or 
delete programs in the pix display 381 by using the mark button 142. If 
no categories have been selected (which is equivalent to selecting the "all 
programs" category), then the pix display 381 displays the 12 programs 
nearest to the last selected program. Similarly, pressing the info button 
136 would display the graphic overlay panel 190 shown in Figure 7 for 
program information. 

Referring now to Figure 34, which illustrates the selection 
of a different pix program to be viewed in the central picture-in-picture 
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It is contemplated that pressing a single, double or triple digit number on 
the numeric keypad 176 , followed by the ok button 178 of the remote 
control 60, will display the corresponding pay-per-view channel, along 
with the 12 nearest channels. It is also contemplated that this pix 
function can be used with audio-visual programs and services other than 
TV shows, in particular when the menu button 172 is used to display the 
output of the VCR 56, the CD 70, a hard disk or one of the other A/V 
devices 57 connected to the A/V transceiver 54. 
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> program sourced from the VCR 56 is displayed in full screen. Thereafter, 
the user can press the play button 168 on the remote control device 60 to 
cause the CPU 63 to instruct the VCR 56 through the A/V connect 
module 66 or the TR control 82 to playback a previously recorded 
r program on the VCR 56. Similarly, the user can press the stop arrow 
buJon 166, the rewind button 164, and/or the fast-forward button 170 on 
the remote control device 60 to further control the program sourced 
from VCR 56. It is contemplated that the CD ROM module 70 and some 
of the other A/V devices 57 including other CDs, VCRs, laser disc 
I/j players, hard disks, telephones, answering machines, as well as 
transmitting devices connected to voice mail, electronic mail and 
various on-line services can be similarly selected and controlled through 
the use of the menu function, as illustrated in Figure 36. 
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corresponding sequence of buttons, as described earlier. It ,s further 
contemplated that the remote control device 60 in conjunction w.th the 
TV 58 and a camera can be used as a video telephone for video 
conferencing and the like, wherein having engaged a telephone line (v,a 
th e menu function, the user speaks direcuy into the remote control 
device 60 as if it were a standard telephone. 
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, such as the title, the production company, a brief description of the 
movie, the rating and the total length. If the user desires to watch the 
pay-per-view movie, he/she depresses the right arrow button 150 once 
again to highlight the ticket icon 388, as illustrated in Figure 41. 

As shown in Figure 41, upon the selection of the 
highlighted "ticket" icon 388, the CPU 63 generates and displays on the 
screen 180 a request for the user to input a "movie pass" password using 
the numeric keypad 176 of the remote control device 60. After entering 
the personal identification number (PIN) using the numeric keypad 176, 
causing a confirmation sound to be played and confirmation symbols 
such as asterisks to fill in a blank confirmation display 425, the user 
depresses the ok button 178 or the select button 155 on the remote control 
device 60 to order and view the pay-per-view movie. As shown in 
Figure 42, transceiver 54 then displays the ordered program if it is 
available, or displays a message informing the viewer that the pay-per- 
view movie will begin screening within X number of minutes. In 
operation, depressing the ok button 178 or the select burton 155 
subsequent to the entering of the movie pass personal identification 
number (PIN) results in the CPU 63 providing a signal to the cable 
service provider 50, over the T/T cable 52 using one of the back channels 
102 (See Figure 3a). It will further be appreciated that during the waiting 
period prior to the transmission of the pay-per-view movie, additional 
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H flmp Shop p^fl Tnterface 



The present invention includes one possible home 
shopping interface, which will now be described. In Figure 43, a channel 
identified as "SHP" or "TV Shop" represents a dedicated home shopping 
service within the electronic spectrum illustrated in Figure 3a. It will be 
noted that although in its present embodiment, TV Shop is presented as 
a television channel, it could also be presented as an on-line service 
through a transmitting device connected to the T/T cable 52. In the 
current embodiment, the selection of the TV Shop service results in a 
copious full-motion video display of various paid-for commercials or 
advertising messages. As shown in the representative screen of Figure 
44, a TV Shop icon is displayed along with a blinking select icon (or right 
arrow icon) in a graphic overlay panel 400, at the start and end of each 
advertisement, prompting users to press the select burton 155 (or right 
arrow 150). 

Referring now to Figure 45, by pressing the select button 155 
(or the right arrow button 150), the user can view additional information 
relating to the advertisements shown in the full motion video section of 
the TV Shop. In the example of Figure 45, it will be noted that a listing of 
the advertisers is provided, in which the advertisers whose commercial 
playing last is highlighted. Through the use of the up arrow burton 
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button 155 (or the right arrow burton 150) to highlight the order icon 409. 
As illustrated in Figure 49, a personal identification order number 
window 420 is displayed in which the user (using the numeric keypad 
176) inputs a personal identification number (PIN). After entering the 
personal identification number (PIN), the ok button 178 or select button 
155 is depressed to order the product. In the presently preferred 
embodiment, once the personal identification number (PIN) of the user 
is entered into the system of the present invention and the ok button 178 
or the select button 155 is depressed, a signal is provided by the CPU 63 to 
the service provider 50, using one of the back channels lu2 (Sec Figure 
3)a. As illustrated in Figure 50, a confirmation of the order, along with a 
delivery time is displayed to the user. It will also be appreciated that 
other home shopping interface variants may be designed and 
implemented using the teachings of the present invention. 
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/ the user as having motion, color, sound and/or animation. Specifically, 
it will be appreciated that the present audio-visual user interface also 
includes an audio interface using a plurality of sound icons and 
backgrounds, which compliment the visual interface and are played 

$ almost every time a function of the present invention is used, and which 
cannot be illustrated graphically here, but are part of the present 
invention. Due to the limitations of a written specification, the reader is 
referred to a videotape entitled "EZTV" submitted by the Applicants 
concurrent with the filing of the application on which this patent is 

lo based. 



WO 95/01058 



PCTAJS94/06993 



77 



provider and said storage means, and managing interaction with service 
offering of said A/V program/service provider. 
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